Fully Non-Parametric Feature Selection Models for Text Categorization

As an important preprocessing technology in text categorization (TC), feature selection can improve the scalability, efficiency and accuracy of a text classifier [1]. In this paper, we propose a two-stage TC model—a fully non-parametric feature selection model. This classification model first selects features using non-parametric statistical methods, and then applies non-parametric classifiers on the selected features for categorization. The improvements are based on two ideas: non-parametric models require no prior knowledge of the data probability distribution, and feature selection could improve classification efficiency and accuracy. We find a competitive fully non-parametric feature selection model: using artificial neural network (ANN) on the features selected by X" statistic (CHI). This model obtains better classification accuracy and requires fewer features than that of the state-of-the-art classifier (support vector machine) on real-world datasets. Therefore, ANN with CHI classification model is good to use on the data with unknown underlying probability distribution. We also discover that two statistical feature selection methods, Kendall rank correlation coefficient and Spearman rank correlation, are strong correlated in TC feature selection.